Simpler PAC-Bayesian bounds for hostile data

PAC-Bayesian learning bounds

with respect to θ ∈ Θ. As we assume that N is potentially very large, we will draw at random some independent identically distributed sample (W1, . . . ,Wn) according to the uniform distribution on { w1, . . . , wN } , where the size n of the statistical sample corresponds to the amount of computations we are ready to make. This sample will be used to choose the parameters. Although in this sce...

On PAC-Bayesian Margin Bounds

Over the past few years, progress has been made in obtaining dimension independent margin bounds. In this note, we revisit the PAC-Bayesian margin bounds proposed by Langford and Shawe-Taylor [4] and later refined by McAllester [6]. In addition to simplifying some of the existing arguments, we use a tighter tail bound on the normal distribution to give an explicit margin bound that is a mild va...

Simplified PAC-Bayesian Margin Bounds

The theoretical understanding of support vector machines is largely based on margin bounds for linear classifiers with unit-norm weight vectors and unit-norm feature vectors. Unit-norm margin bounds have been proved previously using fat-shattering arguments and Rademacher complexity. Recently Langford and Shawe-Taylor proved a dimensionindependent unit-norm margin bound using a relatively simpl...

Combining PAC-Bayesian and Generic Chaining Bounds

There exist many different generalization error bounds in statistical learning theory. Each of these bounds contains an improvement over the others for certain situations or algorithms. Our goal is, first, to underline the links between these bounds, and second, to combine the different improvements into a single bound. In particular we combine the PAC-Bayes approach introduced by McAllester (1...

Perturbation Models and PAC-Bayesian Generalization Bounds